Method and system for combining multiple area-of-interest video codestreams into a combined video codestream

ABSTRACT

A method and system of transmitting a plurality of area-of-interest video codestreams is described. A first video codestream and one or more second video codestreams are generated from a plurality of large format images that are captured. The first video codestream has a first plurality of areas-of-interest selected from the plurality of large format images and the one or more second video codestream have at least a second plurality of areas-of-interest from the same plurality of large format images. The first video codestream is generated at a first frame rate and each of the second video codestreams is generated at a second frame rate. The first and second video codestreams are combined to obtain a combined video codestream. The combined video codestream is then transmitted to a computer system that regenerates the first video codestream and the one or more second video codestreams at their respective frame rates.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.15/444,954, filed on Feb. 28, 2017, which is a continuation of U.S.application Ser. No. 15/192,292, filed on Jun. 24, 2016 (now U.S. Pat.No. 9,621,904), which is a continuation of U.S. application Ser. No.13/232,565, filed on Sep. 14, 2011 (now U.S. Pat. No. 9,407,876), whichclaims the benefit of U.S. Application No. 61/382,823, filed on Sep. 14,2010, the entire contents of these applications being incorporatedherein by reference in their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention pertains to image data management and inparticular to a method and system for encoding and decoding multiplewide-area surveillance area-of-interest video codestreams.

Discussion of Related Art

A very large image generally contains a plurality of pixels, forexample, several hundreds of megapixels (Mp) or several thousands ofmegapixels. Each pixel has one, two or more bands. Each band has acertain color depth or bit depth. For example, an RGB color-based imagehas 3 bands, the red band (R), the green band (G) and the blue band (B).Each of the R, G and B bands can have a depth of 8 bits or more. Hence,in this example, each pixel can have a total bit depth of 24 bits ormore. In another example, an infra-red (IR) image has 1-band, theIR-band. This band can have a bit depth of 12-bits. For the purpose ofcomputational convenience, it can be stored within 16-bits. Hence, inthis example, each pixel can have a total bit depth of 16-bits.

An image sensor can be used to capture a series of images, each imagehaving several hundred megapixels. The images may be captured insequence, for example at a reasonably constant frequency (e.g., 2 Hz).Each image (i.e., each still image) in the sequence or series of imagesmay have one or more distinct bands and may cover any part of theelectromagnetic spectrum that can be captured by the image sensor. Theimage sensor may be a single sensor or a combination or a matrix ofmultiple sensors arranged to generate a single image.

The captured series of images are referred to interchangeably aswide-area surveillance imagery, wide-area motion imagery (WAMI) orwide-area persistent surveillance imagery.

BRIEF SUMMARY OF THE INVENTION

An aspect of the present invention is to provide a method of combining aplurality of area-of-interest (AOI) video codestreams. The methodincludes generating, by a computer server, a plurality of videocodestreams, each video codestream comprising a plurality of AOIs of aplurality of images. The method further includes combining, e.g., by amultiplexer in communication with the computer server, the plurality ofvideo codestreams into a combined video codestream and transmitting thecombined video codestream. A client computer can perform an extractionprocess on the received, combined video codestream, such asdemultiplexing operations, to regenerate the plurality of videocodestreams.

Although the various steps of the method are described in the aboveparagraphs as occurring in a certain order, the present application isnot bound by the order in which the various steps occur. In fact, inalternative embodiments, the various steps can be executed in an orderdifferent from the order described above or otherwise herein.

These and other objects, features, and characteristics of the presentinvention, as well as the methods of operation and functions of therelated elements of structure and the combination of parts and economiesof manufacture, will become more apparent upon consideration of thefollowing description and the appended claims with reference to theaccompanying drawings, all of which form a part of this specification,wherein like reference numerals designate corresponding parts in thevarious figures. In one embodiment of the invention, the structuralcomponents illustrated herein are drawn to scale. It is to be expresslyunderstood, however, that the drawings are for the purpose ofillustration and description only and are not intended as a definitionof the limits of the invention. As used in the specification and in theclaims, the singular form of “a”, “an”, and “the” include pluralreferents unless the context clearly dictates otherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings:

FIG. 1 depicts schematically wide-area surveillance imagery, accordingto an embodiment of the present invention;

FIG. 2 shows schematically an example of an AOI within a very largeimage (W-pixels by H-pixels), according to an embodiment of the presentinvention;

FIGS. 3A and 3B show a schematic representation of a large format image,according to another embodiment of the present invention;

FIG. 4 shows an example of a single video codestream generated from asequence of areas-of-interest (AOIs) from one collection of very largeimages, according to an embodiment of the present invention;

FIG. 5 depicts schematically a plurality of codestreams, according to anembodiment of the present invention;

FIG. 6 is a time diagram of a process for providing a video codestream,according to an embodiment of the present invention;

FIG. 7 is a flow diagram of a method to multiplex multiple videocodestreams into a multiplexed codestream, according to an embodiment ofthe present invention; and

FIG. 8 is a diagram showing a process for generating and transmitting amultiplexed video codestream, according to an embodiment of the presentinvention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

FIG. 1 depicts schematically wide-area surveillance imagery, accordingto an embodiment of the present invention. The wide-area surveillanceimagery comprises a plurality of very large images. Each very largeimage is W-pixels wide by H-pixels tall. In addition each very largeimage has N-bands (where N is an integer greater than 0) and each bandhas a bit-depth B (where B is an integer greater than 0). An example ofvery large images can be images having 10,000 pixel wide (W-pixels) by9,600 pixel tall (H-pixels), a total size of 96 Mp. Another example ofvery large images can be images having 12,000 pixel wide (W-pixels) by12,000 pixel tall (H-pixels), a total size of 144 Mp. Another example ofvery large images can even be images 40,000 pixel wide (W-pixels) and40,000 pixel tall (H-pixels), a total size of 1600 Mp.

The plurality of very large images are collected by one or more sensorsmultiple times per second (H Hz) over a period of time T. H and T aregreater than zero and are real numbers (H, T>0 and H, T Å □ R). A groupof very large images is considered a collection of images. Such verylarge images cannot be practically transmitted or visualized in theirentirety using existing techniques on a display device. Presentcommercial display devices have a pixel width (D_(w)) and a pixel height(D_(H)) that are substantially smaller than the pixel width (W) of theimage and the pixel height (H) of the image, respectively (D_(w)<<W andD_(H)<<H). In addition, current commercial display devices can displayD_(N) bands at a bit-depth of D_(B). The number of bands (D_(N)) in thedisplay device can be the same or different from the number of bands (N)within the image. Similarly, the bit-depth (D_(B)) of each band in thedisplay device can also be the same or different from the bit-depth (B)of a band within the image.

In order to display a large format or size image on a smaller sizedisplay device, the size of the large format image should be reduced,for example, by zooming out of the large format image. However, thisinvolves reducing the number of pixels within the large format image andthus degrading the resolution of the image.

In order to display a large format image at complete pixel size (e.g.,substantially 100% pixel size), an area of interest (AOI) or a viewportmust be extracted from the large format image to be displayed on thedisplay device.

FIG. 2 shows schematically an example of an AOI within a very largeimage (W-pixels by H-pixels), according to an embodiment of the presentinvention. The size of the AOI is substantially the same as the size ofthe display device (D_(w)-pixels by D_(H)-pixels). If the number ofbands (N) within the large format image is different from the number ofbands (DN) of the display device, a conventional method can be appliedto generate (DN) bands from one or more of the (N) bands in the largeformat image to display a portion (AOI) of the large format image on thedisplay device. Similarly, if the bit-depth (B) within the large formatimage is different from the bit-depth (D_(B)) of the display device, aconventional method can be applied to convert the bit-depth (B) into thebit-depth (D_(B)) of the display device so as to display the portion(AOI) of the large format image on the display device.

FIGS. 3A and 3B show a schematic representation of a large format image,according to another embodiment of the present invention. In thisembodiment, the large format image comprises a set of smaller imageshaving a pixel width (S_(w)) and a pixel height (S_(H)). Each smallerimage is substantially smaller in pixel width and pixel height than thelarge format image (S_(w)>0, S_(H)>0, W>>S_(w), H>>S_(H)). The pixelwidth (S_(w)) and the pixel height (S_(H)) of each smaller image can bethe same, less or greater than, respectively, the pixel width of thedisplay (D_(w)) and the pixel height of the display (D_(H)). Eachsmaller image has the same number of bands and the same bit-depth as thelarge format image. Each smaller image is acquired at substantially thesame time as the other smaller images in the same set of smaller imagesof a large format image. Each smaller image can be combined with theother smaller images in the same set of smaller images to create orgenerate a mosaic of images within the large format image, as shown inFIG. 3A or as shown in FIG. 3B.

FIG. 4 shows an example of a single video codestream generated from asequence of AOIs from one collection of very large images, according toan embodiment of the present invention. In order to display or playbacka sequence of AOIs from multiple very large images in a collection(e.g., collection of large format images shown in FIG. 1), the sequenceof AOIs can be encoded into a single video codestream. Each AOI isselected from a corresponding very large image. For example, AOI₁ can beselected from large format image LFI₁, AOI₂ can be selected from largeformat image LFI₂, etc. By arranging AOI₁, AOI₂, etc., in sequence, avideo codestream V can be generated. Although AOI₁, AOI₂, etc. aredescribed being generated as video codestream V in the order the largeformat images LFI₁, LFI₂ are acquired, AOI₁, AOI₂, etc. can also bearranged arbitrarily in any desired order, for example AOI₂ then AOI₁,etc., to be displayed as video codestream V. For example, the videocodestream V can be generated from AOIs (e.g., AOI₁, AOI₂, etc.)extracted from images (LFI₁, LFI₂, etc.) starting at time T₁ and endingat time T₂, where time T₁ may or may not correspond to the time ofgenerating or acquiring LFI₁. Moreover, the video codestream V may ormay not contain AOIs from one or more of the large format images (LFIs).For example, the video codestream V can be generated from AOIs extractedfrom every other captured large format image in the collection of largeformat images. Therefore, the video codestream V is generated at a rateH_(v) Hz that can be equal or different from the rate H Hz of capturingor acquiring the collection of large format images (LFIs).

FIG. 5 depicts schematically a plurality of codestreams V₁, V₂, V₃, . .. , V_(N), according to an embodiment of the present invention. Thecodestreams V₁, V₂, V₃, . . . , V_(N) may have equal number of AOIs ordifferent number of AOIs. In one embodiment, each video codestream V₁,V₂, V₃, . . . V_(N) can be generated from a plurality or sequences ofAOIs within the plurality of very large images. For example, videocodestream V₁ can be generated from plurality or sequence of AOIs atlocation 1 within the plurality of large format images, codestream V₂can be generated from plurality or sequence of AOIs at location 2 withinthe large format images, etc. In addition, each video codestream V₁, V₂,V₃, . . . V_(N) can be generated starting at the same time (i.e., at thesame large format image) or at different times (i.e., at different largeformat images). For example, video codestream V₁ can be generatedstarting by AOI₁ in LFI₁ while video codestream V₂ can be generatedstarting by AOI₂ in LFI₂. Similarly, each video codestream V₁, V₂, V₃, .. . V_(N) can be ended at the same time or at different time.

FIG. 6 is a time diagram of a process for providing a video codestream,according to an embodiment of the present invention. In one embodiment,the process can be implemented, for example, as a web service. In oneembodiment, a user associated with a client computer 10 sends a request12 to a server 14 for wide area motion imagery (WAMI) areas of interest(AOIs). In one embodiment, one or more clients C_(i) (1≤i≤N) may sendthe request to server 14. The request 12 may contain parametersindicating a specific collection of large format images (LFIs) fromwhere the WAMI AOIs will be extracted and a start time T₁ and end timeT₂. Upon receiving the request 12, the server 14 initiates a videocodestream and transmits the WAMI AOIs video codestream at 16 to amultiplexer 18. In one embodiment, multiplexer 18 is provided ashardware or software application within server 14, as depicted in FIG.6. However, as it can be appreciated, in another embodiment, themultiplexer 18 can also be provided as hardware or software applicationoutside the server 14. The server 14 performs this operation formultiple WAMI AOI video codestreams. The multiplexer 18 multiplexes theplurality or multiple video codestreams and transmits at 19 the multiplevideo codestreams as a multiplexed video codestream to demultiplexer 20.In one embodiment, the multiplexed video codestream is continuous andthe format of all multiple video codestreams is the same. In oneembodiment, the multiplexer 18 is continuously multiplexing andtransmitting the multiple video codestreams including the one requestedby C_(i), as a multiplexed video codestream. The demultiplexer 20demultiplexes at 21 the multiplexed or combined video codestream intothe original plurality of video codestreams. In one embodiment, thedemultiplexer 20 is provided as hardware or software application outsidethe client C_(i) 10, as depicted in FIG. 6. In one embodiment, thedemultiplexer 20 is in communication with the client C_(i) 10. In oneembodiment, the client C_(i) 10 receives the i^(th) video codestream andconsumes the video codestream. Each client C_(i) 10 consumes itsrequested video codestream. In one embodiment, if a client C_(i) 10decides to alter a spatial and/or temporal parameter(s) of the AOIs,i.e., change a position of the requested AOIs or a start time T₁ forextracting the AOIs from the LFIs, the client 10 sends a change of therequest of WAMI AOI 22 to the server 14. The server 14 then processesthe change of the request 22 by moving to a new AOI in the collection ofLFIs, the new AOI satisfying the altered spatial and/or temporalparameter(s). In one embodiment, the server 14 while processing thealtered request 22 and updating the AOIs may meanwhile continue sendingor transmitting at 24 the WAMI AOIs to the multiplexer 18 as an i^(th)video codestream. In one embodiment, the server continuously updates theAOIs. In one embodiment, the multiplexer 18 continues multiplexing thevideo codestreams and transmitting at 26 the video codestreams asmultiplexed video codestream. In one embodiment, a frame rate at whichthe i^(th) video codestream is sent need not be the same as a frame rateat which the AOIs are updated within the i^(th) video codestream. In oneembodiment, metadata regarding the AOIs may be embedded within thei^(th) video codestream. For example, metadata can be embedded in avideo codestream as audio, close captioned information, or key lengthvalue (KLV) fields. The multiplexed video codestream is sent todemultiplexer 20 and is demultiplexed at 28 into video codestream forclient consumption. As a result, the client 10 is able to receive asequence of images or video from a different location in the collectionof LFIs as requested by the client 10.

FIG. 7 is a flow diagram of a method to multiplex multiple videocodestreams into a multiplexed codestream, according to an embodiment ofthe present invention. For example, video codestream V₁ may be capturedat a rate of H_(V1) Hz, at 30, V₂ may be captured at a rate of H_(V2)Hz, at 32, V₃ may be captured at a rate of H_(V3) Hz, at 34, . . . andV_(N) may be captured at H_(Vn) Hz, at 36. The capture rates H_(V1),H_(V2), H_(V3), etc. can be equal or different. The video codestreamsV₁, V₂, . . . , V_(N) are multiplexed in a multiplexed video codestreamat a first location (e.g., sender location), at 38. The individual videocodestreams are encoded into a bit rate of the multiplexed videocodestream. Therefore, the bit rate of the original video codestreamsV₁, V₂, . . . , V_(N) may have to be modified. For example, the bit rateof the original video codestreams may be reduced or dialed down so as tofit into the bit rate of the multiplexed video codestream.

For example, if there are five original video codestreams V₁, V₂, . . ., V₅ and each video codestream is at a bit rate of 5 Mbps, 25 Mbps maybe needed to transmit all five video codestreams V₁, V₂, . . . , V₅ as amultiplexed video codestream. However, if only 10 Mbps of bandwidth isavailable for transmitting the multiplexed video codestream, the bitrate of the original video codestreams may need to be modified to “fit”into the 10 Mbps limited bandwidth. If, for example, two of the fiveoriginal video codestreams are very important to the user and thus areset to have the best possible quality as requested by the user while thethree remaining video codestreams are considered by the user to be ofless importance and thus may have a lower quality, the 10 Mbps bandwidthcan be divided into 4 Mbps for the two important video codestreams andthe less important video codestream can be set to a lower bit rate of700 Kbps, 650 Kbps and 650 Kbps. Therefore, while feeding the five videocodestreams, the bit rate of each video codestream can be dynamicallymodified. As a result, the bit rate of each original video codestreamcan be controlled as desired such that the sum of all bit rates of eachof the original video codestream is substantially equal to an allowedbit rate of bandwidth for the multiplexed video codestream.

The multiplexed video codestream can then be transmitted at 40. In oneembodiment, the multiplexed video codestream can be transmitted via link41, such as via cable broadcast channels, fiber optics channels, orwireless channels. At a second location (e.g., receiver location), themultiplexed video codestream is received, at 42. The multiplexed videocodestream can then be demultiplexed, at 44, to regenerate the originalcodestreams V₁ at frame rate H_(V1), at 46, V₂ at frame rate H_(V2), at48, V₃ at frame rate H_(V3), at 50 . . . , and V_(N) at frame rateH_(Vn), at 52. The video codestreams V₁, V₂, V₃, . . . , V_(N) can thenbe played back as wide-area surveillance AOI videos on one or moredisplays. In one embodiment, V₁, V₂, V₃, . . . V_(N) can be played on aplurality of displays D₂, D₂, D₃, . . . D_(N), where V₁ is played on D₁,V₂ is played on D₂, V₃ is played on D₃, . . . and V_(N) is played onD_(N). In another embodiment, V₁, V₂, V₃, . . . V_(N) can be played on anumber of displays smaller than the number of video codestreams. Inwhich case, one or more video codestreams, for example V₁ and V₂, can beplayed on a same display.

For example, by using the present multiplexing scheme to send aplurality of video codestreams and then demultiplexing to reconstructthe original video codestreams, available links or broadcast channelssuch as cable, optical fiber, wireless, etc. can be used fortransmission of the multiplexed video codestream without requiringadditional infrastructure.

FIG. 8 is a diagram showing a process for generating and transmitting amultiplexed video codestream, according to an embodiment of the presentinvention. Upon receiving a request (e.g., a HTTP request) from a client82, the multiplexed video codestream 84 is generated from a sequence orvideo of areas of interest AOIs from one WAMI dataset 86 multiplexedusing multiplexer 87 with other sequences or videos from other areas ofinterest AOIs from other WAMI datasets. The multiplexed video codestream84 is transmitted to demultiplexer 88. The demultiplexer 88 receives themultiplexed video codestream 84 and demultiplexes the multiplexed videocodestream 84 into the original video codestreams so that each videocodestream can be exploited by the client that requested the videocodestream. For example, one or more video codestreams (e.g.,V₁, V₂ andV₃) may be sent to a client 82 that requested these video codestreamswhile other video codestreams (e.g., V₄, V₅, etc. . . . ) may be sent torespective clients that requested the video codestreams. The content andformat of each video codestream through a link 90 between the consumerof the video or user and a producer or server of the video 92 can becontrolled.

In one embodiment, the server 92 of each video codestream is able tochange the AOI 94, and/or the rate at which the AOI 94 is updated intothe video codestream. For example, if the client 82 has “move left,right, up or down” buttons and “zoom in, zoom out” buttons, thesebuttons can be used to modify the AOI 94 that gets displayed in thevideo codestream. Other buttons may also be provided to the user orclient to “flip the AOIs faster or slower” in the video. Thisinformation is conveyed back to the server 92 by the client as one ormore parameters within a request 80. Each client requesting one or morevideo codestreams is able to change its specified AOI 94 and/or the rateat which the specified AOI 94 is updated into the one or more videocodestreams that each client requested. Hence, each client is able tocontrol independently from other clients its requested video codestream.The server or servers 92 can execute the request of each client Cl.

By controlling the AOIs, the client 82 controls the final bit rate ofthe resulting video codestream. For example, for one of several WAMI AOIvideo codestreams being multiplexed by multiplexer 87, if the sourceWAMI is being updated at the rate of 2 frames per second in a 30 FPSvideo code stream, the image update is about only twice a second. As aresult, 15 frames of the video codestream are copies of one frame (oneframe extracted from the two frames per second WAMI). Hence, a lower bitrate can be used while still obtaining a decent video quality since someframes are copies of one or two images. However, if the client requestsfor the AOIs to be updated faster, for example at 15 frames per secondin a 30 fps video, each frame in the video codestream can only duplicatea frame AOI in the WAMI once. As a result, the bit rate of the outputvideo codestream may have to be increased to counterbalance the fasterupdate rate so as not to deteriorate the image video codestream qualityand obtain a good image data for display.

In the 2 fps WAMI to 30 fps video codestream case, a frame in the 2frames per second is repeated fifteen times. That is frame 1 is repeatedfifteen times and frame 2 is also repeated fifteen times. For example,when the 30 fps video codestream is compressed, due to this relativelyhigh redundancy of fifteen copies of a same frame, the frames of theobtained 30 fps video codestream compress well. Therefore, even if onlya lower output bit rate is available, a lot of information can betransmitted in that lower bit rate. On the other hand, in the 15 fpsWAMI to 30 fps video codestream case, one frame is only repeated twiceframe. Hence, a temporal compression to a lower bit rate may degrade thequality of the video codestream. Hence a user may not be able to achieveas good a temporal compression as in the 2 fps to 30 fps case. In orderto make the 30 fps video codestream obtained from the 15 fps WAMI appearas good as the 30 fps video codestream obtained from the 2 fps WAMI, thebit rate of the encoded video codestream may have to be increased.

In one embodiment, the video codestreams can be multiplexed using theISO/IEC 13818-1 standard for multiplexing MPEG-2 transport videocodestreams, as shown at 96. For example, a video codestream can beencoded as an MPEG2 transport stream (MPEG2 TS), as shown at 97. Thevideo MPEG2 TS comprises a program. A description of the program can befound in the ISO/IEC 13818-1 standard. In one embodiment, the programincludes the video codestream of AOIs from WAMI frames, encoded usingthe H.264 codec or MPEG2 codec, key length value or KLV metadataassociated with each WAMI frame, audio codestream, close captioned data,or timing information as required by standard MPEG2 TS, or anycombination of two or more thereof. In one embodiment a plurality ofvideo codestreams that are MPEG2 TS with one program can be multiplexed,as shown at 98. Each video codestream program can be interleaved withprograms from other MPEG2 TS video codestreams to generate a multiplexedMPEG2 TS in accordance with, for example, ISO/IEC 13818-1 standard. Thedemultiplexing process may also be implemented in accordance with ademultiplexing procedure using ISO/IEC 13818-1 standard.

With respect to timing information, each WAMI frame is provided with atime of acquisition. The time of acquisition can be stored as part ofKLV metadata for each V_(i) as shown in FIG. 8. Furthermore, as shown inFIG. 7, each WAMI AOI video stream V_(i) is encoded at a known framerate and bit rate. Therefore, after demultiplexing, the video codestreamcan be played back at the encoded playback rate. The video codestreamcan also be played back at any other playback rate that the clientdesires.

Although in the above description certain types of formats such as MPEG2format, protocols or standards such as ISO/IEC 13818-1 standard arereferred to in the description of some embodiments of the invention, asit can be appreciated the present invention is not in anyway limited tothese formats, procedures, or protocols but can encompass other types offormats, procedures or protocols.

Although the various steps of the method(s) are described in the aboveparagraphs as occurring in a certain order, the present application isnot bound by the order in which the various steps occur. In fact, inalternative embodiments, the various steps can be executed in an orderdifferent from the order described above.

Although the invention has been described in detail for the purpose ofillustration based on what is currently considered to be the mostpractical and preferred embodiments, it is to be understood that suchdetail is solely for that purpose and that the invention is not limitedto the disclosed embodiments, but, on the contrary, is intended to covermodifications and equivalent arrangements that are within the spirit andscope of the appended claims. For example, it is to be understood thatthe present invention contemplates that, to the extent possible, one ormore features of any embodiment can be combined with one or morefeatures of any other embodiment.

Furthermore, since numerous modifications and changes will readily occurto those of skill in the art, it is not desired to limit the inventionto the exact construction and operation described herein. Accordingly,all suitable modifications and equivalents should be considered asfalling within the spirit and scope of the invention.

What is claimed:
 1. A method of providing a combined video codestreambased on multiple areas-of-interest video codestreams from a pluralityof image sources, the method comprising: receiving, by a computersystem, a plurality of requests from one or more client computersystems, each request including parameters indicating: a collection ofimages that includes a corresponding area-of-interest of theareas-of-interest; and an extent of the corresponding one of theareas-of-interest; generating, by the computer system, a plurality ofvideo codestreams, each video codestream corresponding to a respectiveone of the requests and comprising each corresponding area-of-interestover its respective extent wherein a first video codestream of theplurality of codestreams is generated from a plurality of image framesfrom a first image source of the plurality of image sources, the firstvideo codestream comprising a first plurality of areas-of-interestselected from the plurality of image frames, each area-of-interest inthe first plurality of areas-of-interest of the first video codestreambeing selected from a different image frame in the plurality of imageframes, the first video codestream having a first frame rate and asecond video codestream of the plurality of codestreams is generatedfrom the plurality of image frames from a second, different, imagesource of the plurality of image sources, each of the one or more secondvideo codestreams having a second frame rate and comprising a secondplurality of areas-of-interest selected from the plurality of imageframes, each area-of-interest in the second plurality ofareas-of-interest of the respective second video codestream beingselected from a different image frame in the plurality of image frames,the first video codestream and each of the one or more second videocodestream being independent of each other such that informationrepresented in the first video codestream is different from informationrepresented in the second video codestream; multiplexing the pluralityof video codestreams to produce a multiplexed video codestreamcomprising a version of the first video codestream and a version of thesecond video codestream wherein each of each of the first plurality ofareas-of-interest of the first video codestream and each of the secondpluralities of areas-of-interest of the second video code stream have asize substantially equal to a size of a display device of eachrespective requesting client computer system; and transmitting themultiplexed video codestream to the one or more client computer systems,wherein each collection of images comprises a plurality of WAMI imagesand the WAMI images are large format images of at least 10,000 by 9,600pixels, and each area-of-interest comprises a portion of the largeformat image smaller than the large format image.
 2. A method as inclaim 1, wherein the extent of the areas-of-interest comprises aduration.
 3. A method as in claim 1, wherein the extent of theareas-of-interest comprises an image size having fewer pixels thanimages in the collection of images.
 4. A method as in claim 1, whereinthe extent of the areas-of-interest comprises both an image size havingfewer pixels than images in the collection of images and a start and endtime for the images.
 5. A method as in claim 1, wherein one or more ofthe areas-of-interest is continuously updated, and wherein thegenerating, multiplexing, and transmitting includes updating acorresponding video codestream with continuously updated images from theone or more continuously updated areas-of-interest.
 6. A method as inclaim 5, wherein a frame rate at which the continuously updated one ormore of the areas-of-interest is continuously updated is different froma frame rate at which the corresponding video codestream is updated. 7.A method as in claim 1, wherein the multiplexed video codestreamincludes metadata selected from the group consisting of: audio, closecaptioning, key length values, or combinations thereof.
 8. A method asin claim 1, wherein at least one video codestream of the plurality ofvideo codestreams has a frame rate different from at least one othervideo code stream of the plurality of video codestreams.
 9. A method asin claim 8, wherein a total bandwidth of the multiplexed videocodestream is smaller than a sum of bit rates of the plurality of videocodestreams and further comprising: modifying a bit rate of at least oneof the video codestreams prior to the multiplexing based on the totalbandwidth of the multiplexed video codestream.
 10. A method as in claim9, wherein one or more of the plurality of requests includes priorityinformation, and wherein the modifying comprises allocating a largerportion of the bandwidth to requests having a higher priority.
 11. Amethod as in claim 1, wherein transmitting the multiplexed videocodestream comprises transmitting, via a network, the multiplexed videocodestream to a demultiplexer at a location remote from a location ofthe computer system.
 12. A method as in claim 1, wherein the parametersinclude one or more spatial parameters related to the extent of acorresponding one or more of the areas-of-interest, and wherein themethod further comprises adjusting, by the computer system, based on theone or more spatial parameters, a pixel location of the correspondingareas-of-interest during the transmission of the combined videocodestream.
 13. A method as in claim 12, wherein the spatial parameterscomprise parameters derived from a plurality of directional instructionsavailable to a user on the one or more client computers.
 14. A systemfor providing a combined video codestream based on multipleareas-of-interest video codestreams from a plurality of image sources,the method comprising: one or more processors of a computer systemprogrammed to execute one or more computer program instructions that,when executed, cause the one or more processors to: receive, at thecomputer system, a plurality of requests from one or more clientcomputer systems, each request including parameters indicating: acollection of images that includes a corresponding area-of-interest ofthe areas-of-interest, wherein each collection of images comprises aplurality of WAMI images and the WAMI images are large format images ofat least 10,000 by 9,600 pixels, and each area-of-interest comprises aportion of the large format image smaller than the large format image;and an extent of the corresponding one of the areas-of-interest;generate, by the computer system, a plurality of video codestreams, eachvideo codestream corresponding to a respective one of the requests andcomprising each corresponding area-of-interest over its respectiveextent wherein a first video codestream of the plurality of codestreamsis generated from a plurality of image frames from a first image sourceof the plurality of image sources, the first video codestream comprisinga first plurality of areas-of-interest selected from the plurality ofimage frames, each area-of-interest in the first plurality ofareas-of-interest of the first video codestream being selected from adifferent image frame in the plurality of image frames, the first videocodestream having a first frame rate and a second video codestream ofthe plurality of codestreams is generated from the plurality of imageframes from a second, different, image source of the plurality of imagesources, each of the one or more second video codestreams having asecond frame rate and comprising a second plurality of areas-of-interestselected from the plurality of image frames, each area-of-interest inthe second plurality of areas-of-interest of the respective second videocodestream being selected from a different image frame in the pluralityof image frames, the first video codestream and each of the one or moresecond video codestream being independent of each other such thatinformation represented in the first video codestream is different frominformation represented in the second video codestream; multiplex theplurality of video codestreams to produce a multiplexed video codestreamcomprising a version of the first video codestream and a version of thesecond video codestream wherein each of each of the first plurality ofareas-of-interest of the first video codestream and each of the secondpluralities of areas-of-interest of the second video code stream have asize substantially equal to a size of a display device of eachrespective requesting client computer system; and transmit themultiplexed video codestream to the one or more client computer systems.